System, method, and program for predicting information

ABSTRACT

A system includes a learning object storing section that stores objects to be learned, a learning result storing section that stores learning results, and a control section connected to an input section. The control section computes a principal component coefficient vector of a first feature vector of an object to be processed that is designated by the input section, computes a principal component coefficient vector of a second feature vector using a principal component basis vector stored in the learning result storing section, and computes the second feature vector of the object to be processed using the principal component coefficient vector of the second feature vector.

RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 15/524,744, filed May 5, 2017, which is a National Phase entryof PCT Application No. PCT/JP2015/077529, filed Sep. 29, 2015, whichclaims priority from Japanese Patent Application No. 2014-231089, filedNov. 13, 2014, the disclosures of which are hereby incorporated byreference herein in their entirety.

TECHNICAL FIELD

The present disclosure relates to a system, method, and program forpredicting a second feature vector based on a first feature vector.

BACKGROUND ART

The relationship between two pieces of information may be learned, andnew information may be predicted with respect to information to beprocessed using the learning result. For example, in super-resolutiontechnique, a high resolution image is generated from a low resolutionoriginal image. Super-resolution technique has attracted attention asresolution of display devices has been increased, and the technique hasbeen employed in various devices. The super-resolution technique inferslost high frequency components. For example, when enlarging the pixelsize of certain image data vertically and horizontally, an intermediatevalue of two adjacent pixels is computed, and a pixel of theintermediate value fills the gap between the adjacent pixels. Thiscomputing method does not reconstruct lost high frequency components, sothat boundary lines of the enlarged image are blurred. In contrast, thesuper-resolution technique analyzes the pattern of pixels to computepixels that are deemed appropriate, and infers high frequency componentsby filling the gaps between adjacent pixels of the original image withthe computed pixels.

In super-resolution technique, patch super resolution and high frequencycomponent prediction are combined (for example, see Non-Patent Document1). In the technique described in this document, as a method forpredicting high frequency components from low frequency components, thedimensionality of low frequency and high frequency is reduced by PCA,and a high resolution image is predicted by regression analysis. In thiscase, a bilateral filter is used as post-processing.

Techniques for processing data at high speeds with small amounts ofhardware resources have also been considered (see, for example, PatentDocument 1). In the technique described in this document, images to beprocessed are first divided into scenes. Using dictionary tablesapplicable to scenes, lost high frequency components are inferredthrough tree search in the dictionary. In order to speed up the searchon the dictionary table, principal component analysis is used to convertthe index bitmap used as a retrieval key into a first principalcomponent and a second principal component, which are scalar values, andgrouping is performed using the first principal component and the secondprincipal component. Then, the average value of the grouping iscomputed.

-   Patent Document 1: Japanese Laid-Open Patent Publication No.    2013-26659-   Non-Patent Document 1: Columbia University, Wei Liul et al    “Hallucinating Faces: Tensor Patch Super-Resolution and Coupled    Residue Compensation” [online], Columbia University, [retrieved on    Sep. 23, 2014], Internet    <http://www.ee.columbia.edu/˜wliu/CVPR05_LiuWei1.pdf>

SUMMARY OF THE INVENTION

As described above, various attempts have been made in thesuper-resolution technology to achieve higher speed and higher imagequality. For example, in the technique described in Non-Patent Document1, although principal component analysis is performed, high resolutionimages are predicted by regression analysis. In this case, if therelationship between low resolution and high resolution is independentlysubjected to regression learning for each principal componentcoefficient, the overall accuracy may deteriorate due to coexistence ofprincipal component coefficients of high accuracy and principalcomponent coefficients of low accuracy.

In the technique described in Patent Document 1, principal componentanalysis is performed to reduce the amount of dictionary data and tospeed up searches, but variance of principal component coefficients isnot taken into consideration.

Accordingly, it is an objective of the present disclosure to provide asystem, method, and program for efficiently predicting secondinformation, which contains feature vectors, based on first information,which contains feature vectors.

In accordance with one aspect of the present disclosure, a systemconfigured to predict information is provided. The system includes alearning object storing section that stores objects to be learned, alearning result storing section that stores learning results, and acontrol section that is connected to an input section and configured toperform learning processing and prediction processing. In the learningprocessing, the control section computes a first feature vector and asecond feature vector for each object to be learned stored in thelearning object storing section; generates a row vector by dividing aprincipal component coefficient vector of the first feature vector and aprincipal component coefficient vector of the second feature vector byan index representing variation of the principal component coefficientvectors; by performing principal component analysis using the rowvectors of all of the objects to be learned, generates a principalcomponent basis vector based on the principal component coefficientvector of the first feature vector and the principal componentcoefficient vector of the second feature vector; and stores theprincipal component basis vector in the learning result storing section,together with an average vector of the first feature vector and anaverage vector of the second feature vector. In the predictionprocessing, the control section computes a principal componentcoefficient vector of a first feature vector of an object to beprocessed that is designated by the input section; computes a principalcomponent coefficient vector of a second feature vector using theprincipal component basis vector stored in the learning result storingsection, and; using the principal component coefficient vector of thesecond feature vector, computes a second feature vector of the object tobe processed. This allows the second feature vector to be efficientlyand precisely predicted.

In one embodiment, the control section is capable of orthogonalizing theprincipal component basis vector and a component of the first featurevector and storing the orthogonalized vector in the learning resultstoring section. This allows the computation to be speeded up by usingorthogonalization.

In one embodiment, the control section is capable of performingpost-processing, in which the control section compresses the computedhigh-resolution information (the second feature vector), computing thedifference between the compressed high-resolution information and thelow-resolution information of the object to be processed (the firstfeature vector), comparing the difference with a reference value. If thedifference is greater than the reference value, the control sectionincreases the difference and subtracts the increased difference from thehigh resolution information. With this, it is possible to correct errorsthat occur during prediction.

In one embodiment, the index representing the variation may be thestandard deviation of the principal component coefficient vectors. Thisequalizes the scales of the respective factors, maximizing theadvantages of the principal component analysis.

Another aspect of the present disclosure provides a method forpredicting information using the above described system.

Yet another aspect of the present disclosure provides a non-transitorycomputer readable storage medium storing a program for predictinginformation using the above described system.

According to the present disclosure, it is possible to efficientlypredict a second feature vector based on a first feature vector.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an explanatory diagram illustrating an image processing systemaccording to one embodiment.

FIG. 2A is an explanatory diagram illustrating a processing procedure ofthe system of FIG. 1, illustrating first learning processing.

FIG. 2B is an explanatory diagram illustrating a processing procedure ofthe system of FIG. 1, illustrating second learning processing.

FIG. 3 is an explanatory diagram illustrating a processing procedure ofthe system of FIG. 1.

FIG. 4 is an explanatory diagram illustrating a processing procedure ofthe system of FIG. 1.

FIG. 5 is an explanatory diagram illustrating a procedure ofsuper-resolution processing of the system of FIG. 1.

FIG. 6 is an explanatory diagram illustrating a processing procedure ofthe system of FIG. 1.

FIG. 7 is an explanatory diagram illustrating a processing procedure ofthe system of FIG. 1.

FIG. 8 is an explanatory diagram illustrating a processing procedure ofthe system of FIG. 1.

FIG. 9 is an explanatory diagram illustrating a processing procedure ofthe system of FIG. 1.

FIG. 10 is an explanatory diagram illustrating advantages of the systemof FIG. 1.

FIG. 11 is an explanatory diagram illustrating advantages of the systemof FIG. 1.

FIG. 12 is an explanatory diagram illustrating advantages of the systemof FIG. 1.

FIG. 13 is an explanatory diagram illustrating advantages of the systemof FIG. 1.

FIG. 14 is an explanatory diagram illustrating advantages of the systemof FIG. 1.

DESCRIPTION OF THE EMBODIMENTS

A system according to one embodiment of the present disclosure will nowbe described with reference to FIGS. 1 to 14. In the present embodiment,a case will be described where the present disclosure is applied to asuper-resolution technique for increasing the resolution of a facialimage.

As shown in FIG. 1, the present embodiment employs an image processingsystem 20. The image processing system 20 is connected to an inputsection 10 and an output section 15. The input section 10 includes akeyboard and a pointing device, and acquires information input by aperson in charge. The output section 15 includes a display and outputsvarious kinds of information.

The image processing system 20 is a computer system for performingsuper-resolution processing. The image processing system 20 includes acontrol section 21, a face image storing section 22 as a learning objectstoring section, and a learning result storing section 23.

The control section 21 includes control means (a CPU, a RAM, a ROM, andthe like) and performs processing (a learning stage, super-resolutionstage, first prediction stage, second prediction stage, post-processingstage, feature point extraction stage, mesh division stage,normalization stage, feature image extraction stage, principal componentanalysis stage, and the like). By executing programs for respectivestages, the control section 21 functions, as shown in FIG. 1, as alearning section 210, a super-resolution section 215, a first predictionsection 216, a second prediction section 217, a post-processing section218, a feature point extracting section 21 a, a mesh division section 21b, a normalization section 21 c, a feature image extracting section 21d, and a principal component analysis section 21 e.

The learning section 210 performs processing for generating informationnecessary for super-resolution processing using a high-resolution imageand a low-resolution image corresponding to that high-resolution image.

The super-resolution section 215 performs super-resolution processingfor increasing the resolution of images to be processed (input images).The super-resolution section 215 includes a memory that stores images tobe processed and images being computed.

When principal component vectors are orthogonalized, the firstprediction section 216 performs processing for computing an optimalsolution of a high resolution patch.

When the principal component vectors are not orthogonalized, the secondprediction section 217 performs processing for searching for an optimalsolution of a high resolution patch. The second prediction section 217holds data relating to an amount by which a principal componentcoefficient is slightly moved when searching for the optimum solution ofthe principal component coefficient.

The post-processing section 218 performs processing for adjusting errorsusing a generated super-resolution image. In the present embodiment, theiterative back-projection (IBP) process is used. The post-processingsection 218 stores data relating to a threshold value for determiningthe validity of super-resolution results.

The feature point extracting section 21 a performs processing forextracting feature points in images to be processed. In the presentembodiment, feature points (for example, the contour and parts positionsof a face) representing a face image are extracted by patternrecognition.

The mesh division section 21 b performs processing for dividing imagesto be processed into meshes of predetermined shapes based on theextracted feature points. In the present embodiment, triangular meshesare generated using the Delaunay method. The Delaunay method connectspoints in space to generate a group of triangles to maximize the minimumangle of all of the angles of the triangles.

The normalization section 21 c performs processing for transforming themeshes such that the feature points in the image to be processed arearranged in predetermined positions. In the present embodiment, thearrangement of positions on an average face is used for thepredetermined positions. Therefore, the normalization section 21 c holdsdata relating to the arrangement pattern of an average face in advance.Furthermore, the normalization section 21 c performs processing forrestoring the deformed meshes. Thus, the normalization section 21 cstores conversion information at the time of deformation fornormalization.

The feature image extracting section 21 d performs processing forextracting a feature quantity necessary for generating asuper-resolution image from a normalized image to be processed. In thepresent embodiment, a high frequency component of the image is used asthe feature quantity. As discussed below, it is also possible to use theimage to be processed itself (original image) as the feature quantity.

The principal component analysis section 21 e performs principalcomponent analysis processing for obtaining orthogonal vectors fordirections in a plurality of pieces of data in order of magnitude ofvariance in the directions. The principal component analysis section 21e computes eigenvalues and eigenvectors (basis vectors) by principalcomponent analysis processing. The eigenvalues represent variances. Thebasis vectors are arranged in order of the magnitude of variance in thedirections. The lower the order, the less the information held by theprincipal component. Thus, when a certain cumulative contribution ratio(the cumulative of normalized eigenvalues) is reached, the subsequentvectors are excluded from consideration. This limits the dimensionality,so that the computation load will be reduced.

Next, the information stored in each storing section will be described.

The face image storing section 22 stores learning image data relating toa face image (object to be learned) used for learning. The learningimage data is stored when data used to perform learning for thesuper-resolution processing is registered. In this face image data,multiple face images are stored in association with data numbers.

The learning result storing section 23 stores principal component basisvectors and mean vectors. These vectors are generated through couplinglearning. Here, the average vector of an ith coupled patch vector isexpressed as follows.

p _(i) ^((m)) the average vector of the ith coupled patchvector  [Expression 1]

Also, the principal component basis vector of the nth principalcomponent of the ith coupled patch vector is expressed as follows.

U _(in) ^((m)) the nth principal component basis vector of the ithcoupled patch vector  [Expression 2]

Hereinafter, processing in the case where a super-resolution image isgenerated in the image processing system 20 will be described. Thisprocessing is composed of learning processing and super-resolutionprocessing.

Learning Processing

First, the learning processing will be described with reference to FIGS.2A and 2B. The learning processing includes first learning processingand second learning processing. In the first learning processing, pairsof facial images of high resolution and low resolution that have beennormalized so as to match the sizes and positions of face parts aregenerated. In the second learning processing, the pair of facial imagesof high resolution and low resolution are respectively divided intoelements (patches) of a predetermined shape (for example, a rectangle).For each patch, the relationship between low resolution and highresolution is subjected to coupling learning by the principal componentanalysis. First Learning Processing First, the first learning processingwill be described with reference to FIG. 2A.

Here, the control section 21 of the image processing system 20sequentially identifies data to be processed in the face image datastored in the face image storing section 22, and repeats the followingprocessing.

First, the control section 21 performs feature point extractionprocessing (step S1-1). Specifically, the learning section 210 of thecontrol section 21 identifies the outline of the face and feature pointsof face parts using the feature point extracting section 21 a. Thepresent embodiment uses automatic extraction by the active appearancemodel (AAM) method, which is mainly used for tracking facial expressionsand recognizing faces. In the AAM method, a target object (human face)is modeled with a finite number of vertices, and feature points of thetarget object are extracted by fitting the model to the input image.

Subsequently, the control section 21 performs mesh division processing(step S1-2). Specifically, the learning section 210 performs meshdivision on the face image, on which the extracted feature points arearranged, using the mesh division section 21 b.

Next, the control section 21 performs normalization processing (stepS1-3). Specifically, the learning section 210 uses the normalizationsection 21 c to move the grid of each mesh to a predetermined meshposition (for example, an average face), thereby deforming the triangleconstituted by the mesh. Thus, it is possible to generate facial imagedata that contains the feature points of all of the facial images.

This processing is repeated for all of the pieces of face image datastored in the face image storing section 22.

Second Learning Processing

Next, the second learning processing will be described with reference toFIG. 2B.

First, the control section 21 identifies the resolution (low resolution,high resolution) of the object to be processed and repeats the followingprocessing for each resolution.

Here, the control section 21 sequentially identifies data to beprocessed in the face image data stored in the face image storingsection 22, and repeats the following processing.

First, the control section 21 performs resizing processing on the targetimage (step S2-1). Specifically, the control section 21 converts thetarget image into an image size (low-resolution image, high-resolutionimage) to be used in coupling learning discussed below.

Next, the control section 21 performs feature quantity extractionprocessing (step S2-2). Specifically, the learning section 210 extractsa feature quantity necessary for super-resolution from the normalizedtarget image by the feature image extracting section 21 d. Thisprocessing will be discussed below.

Subsequently, the control section 21 performs patch division processing(step S2-3). Specifically, the learning section 210 divides the targetimage into a predetermined number of patches. In the present embodiment,the learning section 210 generates patches of 25×25 pixels.

Next, the control section 21 identifies patches to be processed, andrepeats the following processing for each patch.

In the present embodiment, the control section 21 performs a matrixgenerating processing (step S2-4). Specifically, the learning section210 generates two-dimensional patch data for the patch to be processed.The learning section 210 generates two-dimensional patch data in whichthe RGB values of the patch to be processed are arranged for each pixel.The learning section 210 converts the generated two-dimensional patchdata into a one-dimensional row patch vector (pij). Here, i represents apatch position, and j represents a variable for identifying face imagedata. The learning section 210 stores the one-dimensional row patchvector (pij) in a matrix Di.

Here, a data matrix Dli relating to a low resolution patch is expressedas follows.

$\begin{matrix}{D_{li} = \begin{bmatrix}{\overset{\rightarrow}{p}}_{i\; 1}^{(l)} \\{\overset{\rightarrow}{p}}_{i\; 2}^{(l)} \\{\overset{\rightarrow}{p}}_{i\; 3}^{(l)} \\\vdots\end{bmatrix}} & \left\lbrack {{Expression}\mspace{14mu} 3} \right\rbrack\end{matrix}$

A data matrix Dhi relating to a high resolution patch is expressed asfollows.

$\begin{matrix}{D_{hi} = \begin{bmatrix}{\overset{\rightarrow}{p}}_{i\; 1}^{(h)} \\{\overset{\rightarrow}{p}}_{i\; 2}^{(h)} \\{\overset{\rightarrow}{p}}_{i\; 3}^{(h)} \\\vdots\end{bmatrix}} & \left\lbrack {{Expression}\mspace{14mu} 4} \right\rbrack\end{matrix}$

The control section 21 repeats the matrix generating processing untilall of the patches in the face image data to be processed are processed.

The above processing is repeated until the control section 21 finishesprocessing all of the pieces of face image data.

Next, the control section 21 performs the principal component analysisprocessing (step S2-5). Specifically, the learning section 210 performsprincipal component analysis using the data matrix (Dli, data matrixDhi) for each patch by the principal component analysis section 21 e.

In this case, for low resolution, the following principal componentcoefficient and principal component basis vector are computed.

c _(ijn) ^((l)) the nth principal component coefficient of the ith lowresolution patch vector of data number j  [Expression 5]

{right arrow over (U)} _(in) ^((l)) the nth principal component basisvector of the ith low resolution patch vector  [Expression 6]

For high resolution, the following principal component coefficient andprincipal component basis vector are computed.

c _(ijn) ^((h)) the nth principal component coefficient of the ith highresolution patch vector of the data number j  [Expression 7]

{right arrow over (U)} _(in) ^((h)) the nth principal component basisvector of the ith high resolution patch vector  [Expression 8]

The average vector of the patch vectors is expressed as follows.

p _(i) ^((l)) the average vector of the ith low resolution patchvector  [Expression 9]

p _(i) ^((h)) the average vector of the ith high resolution patchvector  [Expression 10]

In this case, the ith patch vector of the low resolution face image dataj is expressed as follows.

$\begin{matrix}{{\overset{\rightarrow}{p}}_{ij}^{(l)} = {{\overset{\_}{p}}_{i}^{(l)} + {\sum\limits_{n = 1}^{n_{\max{(i)}}}{c_{ijn}{\overset{\rightarrow}{U}}_{in}^{(l)}}}}} & \left\lbrack {{Expression}\mspace{14mu} 11} \right\rbrack\end{matrix}$

Also, the ith patch vector of the high resolution face image data j isexpressed as follows.

$\begin{matrix}{{\overset{\rightarrow}{p}}_{ij}^{(h)} = {{\overset{\_}{p}}_{i}^{(h)} + {\sum\limits_{n = 1}^{n_{\max{(i)}}}{c_{ijn}{\overset{\rightarrow}{U}}_{in}^{(h)}}}}} & \left\lbrack {{Expression}\mspace{14mu}\lbrack 12\rbrack} \right.\end{matrix}$

By limiting nmax(i) to the principal component with the cumulativecontribution ratio of about 98%, the dimensionality can be reduced toabout 100.

The control section 21 repeats the principal component analysisprocessing until completing the processing for all of the resolutions(low-resolution images, high-resolution images).

Subsequently, the control section 21 performs coupling learningprocessing (step S2-6). Specifically, the learning section 210 learnsthe correlation by performing the principal component analysis again foreach patch of the vector obtained by merging the principal componentcoefficients of the low-resolution patch and the high-resolution patch.This processing will be discussed below.

Feature Quantity Extraction Processing

Next, with reference to FIG. 3, the feature quantity extractionprocessing (step S2-2) will be described.

First, the control section 21 performs processing for reducing theoriginal image (step S3-1). Specifically, the feature image extractingsection 21 d of the control section 21 compresses the face image to beprocessed, thereby reducing the size of the image. As a result, highfrequency components of the face image are eliminated.

Next, the control section 21 performs processing for computing a lowfrequency component (step S3-2). Specifically, the feature imageextracting section 21 d enlarges the reduced face image data to theoriginal size. This generates image data of the original size composedof a low frequency component (a low frequency component image).

Next, the control section 21 performs processing for computing a highfrequency component (step S3-3). Specifically, the feature imageextracting section 21 d extracts a high frequency component bysubtracting the low frequency component image from the original faceimage.

Coupling Learning Processing

Next, the coupling learning processing will be described with referenceto FIG. 4. The principal component coefficient of the low resolutionpatch is coupled to the principal component coefficient of the highresolution patch to generate one row vector (one dimensional vector).The generated row vector is stored in a matrix, and the principalcomponent analysis is performed again. By coupling the information ofthe low resolution patch and the information of the high resolutionpatch to perform the principal component analysis, the relationshipbetween the low resolution patch and the high resolution patch islearned.

First, the control section 21 performs processing for generating a rowvector (step S4-1). Specifically, the learning section 210 generates avector P (m)ij, which is a combination of the principal componentcoefficient of the low-resolution patch and the principal componentcoefficient of the high-resolution patch.

$\begin{matrix}{{\overset{\rightarrow}{p}}_{ij}^{(m)} = \left( {\frac{c_{{ij}\; 0}^{(l)}}{s_{i\; 0}^{(l)}},\frac{c_{{ij}\; 1}^{(l)}}{s_{i\; 1}^{(l)}},{\ldots\mspace{14mu}\frac{c_{{ijn}_{\max\;{({l,i})}}}^{(l)}}{s_{i\; n_{\max\;{({l,i})}}}^{(l)}}},\frac{c_{{ij}\; 0}^{(h)}}{s_{i\; 0}^{(h)}},\frac{c_{{ij}\; 1}^{(h)}}{s_{i\; 1}^{(h)}},{\ldots\mspace{14mu}\frac{c_{{ijn}_{\max\;{({h,i})}}}^{(h)}}{s_{i\; n_{\max\;{({h,i})}}}^{(h)}}}} \right)} & \left\lbrack {{Expression}\mspace{14mu} 13} \right\rbrack\end{matrix}$

Here, Sin represents scaling factors shown below.

s _(in) ^((l)) the scaling factor of the nth principal componentcoefficient of the ith low resolution patch vector  [Expression 14]

s _(in) ^((h)) the scaling factor of the nth principal componentcoefficient of the ith high resolution patch vector  [Expression 15]

For the scaling factor Sin, the standard deviation of the nth principalcomponent coefficient Cijn of the ith low-resolution patch vector isused.

Subsequently, the control section 21 performs processing for generatinga matrix (step S4-2). Specifically, the learning section 210 generates adata matrix Dmi of a coupled patch vector using the one-dimensional rowpatch vector Pij. The data matrix Dmi is expressed as follows.

$\begin{matrix}{D_{mi} = \begin{bmatrix}{\overset{\rightarrow}{p}}_{i\; 1}^{(m)} \\{\overset{\rightarrow}{p}}_{i\; 2}^{(m)} \\{\overset{\rightarrow}{p}}_{i\; 3}^{(m)} \\\vdots\end{bmatrix}} & \left\lbrack {{Expression}\mspace{14mu} 16} \right\rbrack\end{matrix}$

Next, the control section 21 performs processing for analyzing theprincipal component for each patch (step S4-3). Specifically, theprincipal component analysis section 21 e of the control section 21performs principal component analysis using the data matrix Dmi.

In this case, the following principal component coefficient andprincipal component basis vector are computed.

c _(ijn) ^((m)) the nth principal component coefficient of the ithcoupled patch vector of the data number j  [Expression 17]

{right arrow over (U)} _(in) ^((m)) the nth principal component basisvector of the ith coupled patch vector  [Expression 18]

The principal component basis vector Uin is expressed as follows.

$\begin{matrix}{{\overset{\rightarrow}{U}}_{in}^{(m)} = \left\{ {{\overset{\rightarrow}{U}}_{in}^{({m,l})}{\overset{\rightarrow}{U}}_{in}^{({m,h})}} \right\}} & \left\lbrack {{Expression}\mspace{14mu} 19} \right\rbrack \\{{{\overset{\rightarrow}{U}}_{i\; n}^{({m,l})}\mspace{14mu}{the}\mspace{14mu}{low}\text{-}{resolution}\mspace{14mu}{part}\mspace{14mu}{of}\mspace{14mu}{the}\mspace{14mu}{first}\mspace{14mu}{half}}\text{}{{of}\mspace{14mu}{\overset{\rightarrow}{U}}_{i\; n}^{(m)}}} & \left\lbrack {{Expression}\mspace{14mu} 20} \right\rbrack \\{{{\overset{\rightarrow}{U}}_{i\; n}^{({m,h})}\mspace{14mu}{the}\mspace{14mu}{high}\text{-}{resolution}\mspace{14mu}{part}\mspace{14mu}{of}\mspace{14mu}{the}\mspace{14mu}{second}}\text{}{{half}\mspace{14mu}{of}\mspace{14mu}{\overset{\rightarrow}{U}}_{i\; n}^{(m)}}} & \left\lbrack {{Expression}\mspace{14mu} 21} \right\rbrack\end{matrix}$

Next, the control section 21 performs orthogonalization processing (stepS4-4). Specifically, the learning section 210 orthogonalizes thelow-resolution component (Expression 20) by the Gram-Schmidt process. Inthis case, the high-resolution component (Expression 21) is alsoconverted using the orthogonalization factor in the low-resolutioncomponent.

Next, the control section 21 performs processing for determining whetherorthogonalized vectors can be used (step S4-5). Specifically, thelearning section 210 checks whether there will be no problem in terms ofaccuracy even if the orthogonalization is performed to limit the numberof principal component vectors to nmax (l, i). Specifically, thelearning section 210 computes the cumulative contribution ratio of thehigh-resolution principal component vectors when the number of theprincipal component vectors is limited to nmax (1, i). When thecumulative contribution ratio has reached a predetermined value (forexample, 98%), the learning section 210 determines that theorthogonalized vectors can be used.

When there is no problem in terms of accuracy and it is determined thatthe orthogonalized vectors can be used (YES in step S4-5), the controlsection 21 performs processing for storing the orthogonalized coupledvectors and the average vector (Step S4-6). Specifically, the learningsection 210 stores the orthogonalized coupled vectors and the averagevector in the learning result storing section 23.

In contrast, when there is a problem in terms of accuracy and it isdetermined that the orthogonalized vectors cannot be used (NO in stepS4-5), the control section 21 performs processing for storing theoriginal coupled vectors and the average vector (Step S4-7).Specifically, the learning section 210 stores the original coupledvector and the average vector in the learning result storing section 23.

The average vector of the coupled patch vector is expressed as follows.

p _(i) ^((m)) the average vector of the ith coupled patchvector  [Expression 22]

In this case, the ith one-dimensional row patch vector Pij of the data jis expressed as follows.

$\begin{matrix}{{\overset{\rightarrow}{p}}_{i,j}^{(m)} = {{\overset{\_}{p}}_{i}^{(m)} + {\sum\limits_{n = 1}^{n_{\max{({m,i})}}}{c_{i,j,n}^{(m)}{\overset{\rightarrow}{U}}_{in}^{(m)}}}}} & \left\lbrack {{Expression}\mspace{14mu} 23} \right\rbrack \\{{\overset{\rightarrow}{U}}_{in}^{(m)} = \left\{ {{\overset{\rightarrow}{U}}_{in}^{({m,l})}{\overset{\rightarrow}{U}}_{in}^{({m,h})}} \right\}} & \left\lbrack {{Expression}\mspace{14mu} 24} \right\rbrack\end{matrix}$

Super-Resolution Processing

Next, with reference to FIGS. 5 to 9, the super-resolution processingwill be described. First, with reference to FIG. 5, the outline of thesuper-resolution processing will be described.

From an input image, a patch vector V10 consisting of the principalcomponent coefficient of a low resolution patch is acquired. Alow-resolution part V21 of the principal component basis vector of thecoupled patch vector and a low-resolution part V22 of the average vectorof the coupled patch vector are acquired from the learning resultstoring section 23. A principal component coefficient Cij is computedthat expresses the patch vector V10 with the low-resolution part V22 ofthe average vector and the low-resolution part V21 of the principalcomponent basis vector.

A high-resolution part V31 of the principal component basis vector ofthe coupled patch vector and a low-resolution part V32 of the averagevector of the coupled patch vector are acquired from the learning resultstoring section 23. A super-resolution of the input image is generatedusing the computed principal component coefficient Cij, thehigh-resolution part V31 of the principal component basis vector, andthe high-resolution portion V32 of the average vector.

Next, with reference to FIG. 6, the super-resolution processing will bedescribed in concrete terms.

First, the control section 21 performs processing for inputting a faceimage (step S5-1). Specifically, the super-resolution section 215 of thecontrol section 21 acquires an input image designated by the inputsection 10.

Next, the control section 21 performs processing for extracting facefeature points (step S5-2). Specifically, the super-resolution section215 extracts feature points using the feature image extracting section21 d in the same manner as in step S1-1.

Subsequently, the control section 21 performs mesh division processing(step S5-3). Specifically, the super-resolution section 215 performsmesh division on the face image using the mesh division section 21 b inthe same manner as in step S1-2.

Next, the control section 21 performs normalization processing (stepS5-4).

Specifically, the super-resolution section 215 moves the grid of eachmesh to a predetermined mesh position (for example, an average face),and deforms the triangle constituted by the mesh in the same manner asin step S1-3.

Next, the control section 21 performs feature quantity extractionprocessing (step S5-5). More specifically, the super-resolution section215 extracts the feature quantity necessary for super-resolution fromthe normalized input image by the feature image extracting section 21 din the same manner as in step S2-2.

Subsequently, the control section 21 performs patch division processing(step S5-6). Specifically, the super-resolution section 215 divides theinput image into a predetermined number of patches in the same manner asin step S2-3.

Next, the control section 21 identifies patches to be processed, andrepeats the following processing for each patch.

In this embodiment, the control section 21 performs processing forpredicting a super-resolution patch (step S5-7). Specifically, thesuper-resolution section 215 performs first prediction processing orsecond prediction processing for the super-resolution patch. When theprincipal component vectors are orthogonalized, the super-resolutionsection 215 performs first prediction processing. When the principalcomponent vectors are not orthogonalized, the super-resolution section215 performs second prediction processing. The processing will bediscussed below.

The super-resolution section 215 repeats the above processing until allof the patches of the image to be processed are processed.

Subsequently, the control section 21 performs post-processing (stepS5-8). This processing will be discussed below.

Next, the control section 21 performs processing for restoring thenormalized image to the shape of the original image (step S5-9).Specifically, the super-resolution section 215 moves the grid of eachmesh in a direction opposite to the moving direction of step S5-4,thereby restoring the arrangement of the grid of each mesh.

First Prediction Processing

With reference to FIG. 7, the first prediction processing for the superresolution patch will be described. This processing is performed whenthe principal component vectors are orthogonalized and it is desired tospeed up the processing.

First, the control section 21 performs inner product processing (stepS6-1). Specifically, the first prediction section 216 of the controlsection 21 computes the inner product using the following expression.

c _(i,t,n) ^((m)) ={right arrow over (p)} _(i,t) ^((,l)) ·{right arrowover (U)} _(in) ^((m,l))  [Expression 25]

Here, Uin and Pit are as follows.

{right arrow over (U)} _(in) ^((m,l)) the low-resolution part of theorthogonalized coupled principal component vector  [Expression 26]

{right arrow over (p)} _(i,t) ^((l)) the low-resolution patchcoefficient vector obtained from the input  [Expression 27]

Next, the control section 21 performs processing for computing thehigh-resolution patch (step S6-2). Specifically, the first predictionsection 216 generates a high-resolution patch coefficient vector in eachpatch using the following expression and stores the high-resolutionpatch coefficient vector in the memory of the super-resolution section215.

$\begin{matrix}{{\overset{\rightarrow}{p}}_{i,t}^{({m,h})} = {{\overset{\_}{p}}_{i}^{({m,h})} + {\sum\limits_{n = 1}^{n_{\max{({m,i})}}}{c_{i,t,n}^{(m)}{\overset{\rightarrow}{U}}_{in}^{({m,h})}}}}} & \left\lbrack {{Expression}\mspace{14mu} 28} \right\rbrack\end{matrix}$

Second Prediction Processing

With reference to FIG. 8, the second prediction processing for the superresolution patch will be described. This processing is performed whenthe principal component vector is not orthogonalized or when it isdesired to consider principal component vectors of a number greater thanor equal to nmax (1, i).

First, the super-resolution section 215 performs processing forcomputing the low-resolution patch coefficient vector by principalcomponent analysis (step S7-1). Specifically, the super-resolutionsection 215 computes the low-resolution patch coefficient vector usingthe principal component analysis section 21 e.

{right arrow over (p)} _(i,t) ^((l)) the low-resolution patchcoefficient vector obtained from the input  [Expression 29]

Subsequently, the control section 21 performs processing forprovisionally setting the principal component coefficient (step S7-2).Specifically, the second prediction section 217 of the control section21 computes an inner product to compute the principal componentcoefficient Ci,t,n of the input image (data number: t) as in step S6-1.Since the principal component vector is not orthogonalized, the computedprincipal component coefficient Ci,t,n is provisionally set as aninitial value, and the optimum solution search, which will be discussedbelow, is performed.

Next, the control section 21 performs processing for computing thelow-resolution patch coefficient vector based on the provisionally setprincipal component coefficient (step S7-3). Specifically, the secondprediction section 217 acquires the following average vector andprincipal component basis vector from the learning result storingsection 23.

p _(i) ^((m,l)) the low-resolution part of the first half of p _(i)^((m))  [Expression 30]

{right arrow over (U)} _(in) ^((m,l)) the low-resolution part of thefirst half of {right arrow over (U)} _(in) ^((m))  [Expression 31]

In this case, the low-resolution component (the low-resolution patchcoefficient vector) of the coupled vector in each patch of thelow-resolution image of the input image (data number: t) is expressed asfollows using the provisionally set principal component coefficientCi,t,n.

$\begin{matrix}{{\overset{\rightarrow}{p}}_{i,t}^{({m,l})} = {{\overset{\_}{p}}_{i}^{({m,l})} + {\sum\limits_{n = 1}^{n_{\max{({m,i})}}}{c_{i,t,n}^{(m)}{\overset{\rightarrow}{U}}_{in}^{({m,l})}}}}} & \left\lbrack {{Expression}\mspace{14mu} 32} \right\rbrack\end{matrix}${right arrow over (p)} _(i,t) ^((m,l)) the low-resolution part of thefirst half of the vector in which the principal component coefficient ofthe low-resolution patch and the principal component coefficient of thehigh-resolution patch are coupled  [Expression 33]

Next, the control section 21 performs processing for computing adifference (step S7-4). Specifically, using the following expression,the second prediction section 217 computes the difference between thelow-resolution patch coefficient vector computed in step S7-1 and thelow-resolution patch coefficient vector computed in step S7-3.

E=|{right arrow over (p)} _(i,t) ^((m,l)) −{right arrow over (p)} _(i,t)^((l))|[Expression 34]

The second prediction section 217 provisionally stores the computeddifference in the memory in association with the provisionally set lowresolution patch coefficient vector.

Next, the control section 21 performs processing for determining whetherrepetition has been ended (step S7-5). Specifically, when the differencereaches the minimum value, the second prediction section 217 determinesthat the repetition has been ended.

If the difference is still decreasing and it is determined that therepetition has not been ended yet (in the case of NO in step S7-5), thecontrol section 21 slightly moves the position of the provisionally setprincipal component coefficient in a direction of a low gradient, andrepeats the processing in and after step S7-3.

In contrast, when the difference has reversed to an increasing trend andit is determined that the repetition has been ended (YES in step S7-5),the control section 21 performs processing for identifying the principalcomponent coefficient of the minimum value of the difference (stepS7-6). Specifically, the second prediction section 217 identifies theprincipal component coefficient at which the difference has the minimumvalue.

Next, the control section 21 performs processing for computing thehigh-resolution patch (step S7-7). Specifically, the second predictionsection 217 acquires the following average vector and principalcomponent basis vector from the learning result storing section 23.

{right arrow over (p)} _(ij) ^((m,l)) the high-resolution part of thesecond half of {right arrow over (p)} _(ij) ^((m))  [Expression 35]

{right arrow over (U)} _(in) ^((m,h)) the high-resolution part of thesecond half of {right arrow over (U)} _(in) ^((m))  [Expression 36]

The second prediction section 217 computes the high-resolution patchcoefficient vector by applying the principal component coefficienthaving the minimum value of the difference to the following expression,and stores the high-resolution patch coefficient vector in the memory ofthe super-resolution section 215.

$\begin{matrix}{{\overset{\rightarrow}{p}}_{i,t}^{({m,h})} = {{\overset{\_}{p}}_{i}^{({m,h})} + {\sum\limits_{n = 1}^{n_{\max{({m,i})}}}{c_{i,t,n}^{(m)}{\overset{\rightarrow}{U}}_{in}^{({m,h})}}}}} & \left\lbrack {{Expression}\mspace{14mu} 37} \right\rbrack\end{matrix}$

Post-Processing

The post-processing will be described with reference to FIG. 9. Here,using the iterative back projection (IBP) method, the error between theinput image and the image obtained when the computed super-resolution isreduced to the size of the input image is diminished.

First, the control section 21 performs processing for acquiring thesuper-resolution result (step S8-1). Specifically, the super-resolutionsection 215 invokes an image composed of the high-resolution patchcoefficient vector stored in the memory of the super-resolution section215 by the post-processing section 218.

Next, the control section 21 performs processing for reducing thesuper-resolution result (step S8-2). Specifically, the post-processingsection 218 of the control section 21 generates a compressed imageobtained by reducing the invoked super-resolution image to the size ofthe input image.

Next, the control section 21 performs processing for computing thedifference from the original image (step S8-3). Specifically, thepost-processing section 218 compares the compressed image with the inputimage and computes the difference.

Next, the control section 21 performs processing for determining whetherthe difference is less than or equal to a threshold value (step S8-4).Specifically, the post-processing section 218 compares the computeddifference with an allowable threshold value.

When determining that the difference is less than or equal to thethreshold value (YES in step S8-4), the control section 21 performsprocessing for outputting the super-resolution result (step S8-5).Specifically, the post-processing section 218 outputs thesuper-resolution image stored in the memory as the super-resolutionprocessing result.

When determining that the difference is not less than or equal to thethreshold value (NO in step S8-4), the control section 21 performsprocessing for increasing the difference (step S8-6). Specifically, thepost-processing section 218 generates a difference image by increasingthe computed difference to the resolution of the super-resolution image.

Next, the control section 21 performs processing for correcting thesuper-resolution result (step S8-7). Specifically, the post-processingsection 218 generates an image obtained by subtracting the differenceimage from the image stored in the memory, and stores the image in thememory to update the image.

The control section 21 repeats the processing after the reductionprocessing of the super-resolution result (step S8-2).

The present embodiment achieves the following advantages.

(1) In the present embodiment, the coupling learning by the principalcomponent analysis is performed using a low-resolution image and ahigh-resolution image. If the relationship between the low resolutionand the high resolution is independently learned for each principalcomponent coefficient by regression analysis, the overall accuracy maydeteriorate due to coexistence of principal component coefficients ofhigh accuracy and principal component coefficients of low accuracy. Incontrast, in the present embodiment, since the learning is performed foreach coefficient vector, the accuracy of the entire principal componentcoefficient is consistent, and the overall accuracy is improved.

Compared to the method of searching an image database, high-resolutionimages are generated at high speed by excluding principal componentvectors having low cumulative contribution ratios to reduce thedimensionality. Furthermore, the data volume is markedly reduced ascompared with the method of searching an image database. In addition,since the learning processing is performed using a plurality of images,it is possible to generate a high-quality high-resolution image.

As shown in FIG. 10, jaggies occur when an input image is enlarged, buthigh resolution can be reproduced in the super-resolution resultobtained by the super-resolution processing of this embodiment.Resolution close to a correct answer, which is the original image (highresolution) of the low resolution input image, can be obtained.

(2) In the present embodiment, the control section 21 performsprocessing for extracting a feature quantity (step S2-2). It is possibleto compute the feature quantity using a high frequency component andobtain comparatively satisfactory super resolution.

FIG. 11 shows comparison between the case of using an original image toextract feature quantities and the case of using a high frequencycomponent to extract feature quantities. If the original image is usedin both of the computation of the low-resolution feature quantity andthe computation of the high-resolution feature quantity, block artifactsappear in contour portions. In contrast, when a high frequency componentis used in either the computation of the low-resolution feature quantityor the high-resolution feature quantity, favorable super-resolution isobtained.

(3) In the present embodiment, for the scaling factor Sin, the standarddeviation of the nth principal component coefficient Cijn of the ithlow-resolution patch vector is used. Normally, when principal componentanalysis is performed by coupling two different pieces of information(feature vectors), one of the feature vectors is multiplied by a scalingfactor in order to absorb a greater amount of difference (difference inscale) between the dimensions of the two feature vectors before beingcoupled. If the principal component analysis is performed using ascaling factor of a fixed value, the principal component coefficientCijn of the high-frequency component of a large number would be drownedout by variance of the principal component coefficient Cijn of thelow-frequency component of a small number. In contrast, by performingthe principal component analysis after performing division by thestandard deviation, the scales of the principal component coefficientsbecome equal, maximizing the advantages of the principal componentanalysis.

In a conventional method that uses no scaling factor, thesuper-resolution images deteriorates as shown in FIG. 12, but thepresent embodiment achieves a favorable super-resolution.

(4) In the present embodiment, the control section 21 performs the firstprediction processing or the second prediction processing for thesuper-resolution patch. As a result, depending on the state oforthogonalization, two prediction processings can be used selectively.

FIG. 13 shows an example of comparison between a case of using the firstprediction processing (optimum solution search) and a case of using thesecond prediction processing (orthogonalization). In both cases, littledifference occurs between the correct image and the super-resolutionimage. In contrast, the computation time is shortened byorthogonalization, and high-speed processing equivalent to regressionanalysis is achieved.

(5) In the present embodiment, the control section 21 performs thepost-processing. This suppresses errors.

When block artifacts are generated as shown in FIG. 14, thepost-processing reduces the influence.

The above-described embodiment may be modified as follows.

In the above illustrated embodiment, the super resolution processing ofa face image is performed using a low resolution vector as the firstfeature vector and a high resolution vector as the second featurevector. The object of the super-resolution processing is not limited toface images, but the processing may be applied to super resolution forconverting an object having a pattern of shape to high image quality. Inthis case, a learning processing is performed using a low-resolutionimage and a high-resolution image having specific patterns.

Furthermore, the method of the present disclosure can be applied notonly to super-resolution but also to various usages for estimatingsecond vector data from first vector data. For example, in imagesegmentation, latent information is extracted from explicit information.

In two-dimensional image feature point recognition, an objecttwo-dimensional image is used as the first feature vector, and a featurepoint of the object image is used as the second feature vector.

In image segmentation of a two-dimensional image, an objecttwo-dimensional image is used as the first feature vector, andsegmentation of the object image is used as the second feature vector.

In three-dimensional image feature point recognition, an objectthree-dimensional image is used as the first feature vector, and afeature point of the object image is used as the second feature vector.

In image segmentation of a three-dimensional image, an objectthree-dimensional image is used as the first feature vector, andsegmentation of the object image is used as the second feature vector.

Specific examples of information prediction will be described below.

For example, in correcting blurred or out-of-focus images, an imagevector of a blurred or out-of-focus image is used as the first featurevector, and an image vector of an image from which blurring and wrongfocus have been corrected is used as the second feature vector.

In security camera dynamic image analysis (action prediction),time-series data of actions of a person until the start of prediction isused as the first feature vector, and time-series data of actions of theperson after the start of the prediction is used as the second featurevector.

Also, the present disclosure can be applied to prediction of informationother than images.

For example, in order to improve the quality of sound, a low qualitysound signal is used as the first feature vector, and a high qualitysound signal is used as the second feature vector. In this case, thesignal itself of the sound signal itself or a vector having a highfrequency component as an element is used.

In meteorological prediction, time-series weather data until the startof the prediction is used as the first feature vector, and time-seriesweather data after the start of the prediction is used as the secondfeature vector. In this case, a vector is used that includes, aselements, various kinds of meteorological data (weather, pressure,temperature, humidity, wind direction, wind speed, and the like).

In economic prediction, time-series data of stock prices and economicindicators until the start of the prediction is used as the firstfeature vector, and time-series data of stock prices and economicindicators after the start of the prediction is used as the secondfeature vector. In this case, a vector is used that includes, aselements, stock prices and economic indicators (stock prices of variousissues, interest rates, business conditions, employment statistics,inflation rate, trade environment and the like).

In health prediction, time-series biomarker data until the start of theprediction is used as the first feature vector, and time-seriesbiomarker data after the start of the prediction is used as the secondfeature vector. In this case, a vector is used that includes, aselements, various biomarkers (body weight, height, health diagnosisvalue, and the like).

In traffic prediction, time-series traffic data until the start of theprediction is used as the first feature vector, and time-series trafficdata after the start of the prediction is used as the second featurevector. In this case, a vector is used that includes, as elements,various kinds of traffic data (sites of observation, number of cars,types of cars, car speeds, destinations, and the like).

In consumption behavior prediction, time-series consumption behaviordata until the start of the prediction is used as the first featurevector, and time-series consumption behavior data after the start of theprediction is used as the second feature vector. In this case, a vectoris used that includes personal action data (names of articles, amount ofconsumption, various economic indicators, and the like) as elements.

In the above illustrated embodiment, linear principal component analysisis performed, but it is also possible to perform nonlinear kernelprincipal component analysis.

In the above illustrated embodiment, for the scaling factor Sin, thestandard deviation of the nth principal component coefficient Cijn ofthe ith low-resolution patch vector is used. The scaling factor Sin isnot limited to standard deviation. For example, an index representingvariation of the principal component coefficient Cijn can be used.

In the above illustrated embodiment, for the scaling factor Sin, thestandard deviation of the nth principal component coefficient Cijn ofthe ith low-resolution patch vector is used. The scaling factor Sin isnot limited to standard deviation. For example, a statistical variablerepresenting variance of the principal component coefficient Cijn can beused.

1. A system configured to consistently and accurately predict information, comprising: a learning object storage device that stores objects to be learned; a learning result storage device that stores learning results; and a control section including a processor and connected to an input section, configured to perform: learning processing in which the control section: computes a first feature vector and a second feature vector for each object to be learned stored in the learning object storage device, the first feature vector being a feature vector of first information and the second feature vector being a feature vector of second information, the second information being a result of prediction from the first information, generates a coupled basis vector by: generating a coupled vector for each object to be learned by coupling the first feature vector and the second feature vector after equalizing scales of the first feature vector and the second feature vector, and then performing machine learning using the coupled vectors of all of the objects to be learned, so as to generate a coupled basis vector, and stores the coupled basis vector in the learning result storage device, and prediction processing, in which the control section computes a first feature vector of an object to be processed that is designated by the input section and computes a coupling coefficient of the first feature vector of the object to be processed, and using the coupling coefficient of the first feature vector and the coupled basis vector that is stored in the learning result storage device, computes a second feature vector of the object to be processed.
 2. The system according to claim 1, wherein the control section is configured to generate the first feature vector by dividing a first coefficient, which represents the first information by a first basis vector, by a first index that represents variation of the first information, and to generate the second feature vector by dividing a second coefficient, which represents the second information by a second basis vector, by a second index that represents variation of the second information.
 3. The system according to claim 1, wherein the control section is configured to perform post-processing, in which the control section converts the computed second feature vector, computes a difference between the converted second feature vector and the first feature vector of the object to be processed, compares the difference with a reference value, and when the difference is greater than the reference value, converts the difference and subtracts the converted difference from the second feature vector.
 4. The system according to claim 1, wherein generating the coupled basis vector comprises multiplying one of the first feature vector and the second feature vector by a scaling factor.
 5. The system according to claim 1, wherein the first index is a standard deviation of a coupling coefficient of the first feature vector and the second index is a standard deviation of a coupling coefficient of the second feature vector.
 6. The system according to claim 1, wherein the first information is a low-resolution image and the second information is a high-resolution image.
 7. A method for consistently and accurately predicting information performed by an information predicting computer system that includes a learning object storage device that stores objects to be learned, a learning result storage device that stores learning results, and a control section including a processor and connected to an input section, the method causes the control section to perform: learning processing in which the control section computes a first feature vector and a second feature vector for each object to be learned stored in the learning object storage device, the first feature vector being a feature vector of first information and the second feature vector being a feature vector of second information, the second information being a result of prediction from the first information, generates a coupled basis vector by: generating a coupled vector for each object to be learned by coupling the first feature vector and the second feature vector after equalizing scales of the first feature vector and the second feature vector, and then performing machine learning using the coupled vectors of all of the objects to be learned, so as to generate a coupled basis vector, and stores the coupled basis vector in the learning result storage device, and prediction processing, in which the control section computes a first feature vector of an object to be processed that is designated by the input section, and computes a coupling coefficient of the first feature vector of the object to be processed, and using the coupling coefficient of the first feature vector and the coupled basis vector that is stored in the learning result storage device, computes a second feature vector of the object to be processed.
 8. The method according to claim 7, further comprising, by the control section generating the first feature vector by dividing a first coefficient, which represents the first information by a first basis vector, by a first index that represents variation of the first information, and generating the second feature vector by dividing a second coefficient, which represents the second information by a second basis vector, by a second index that represents variation of the second information.
 9. The method according to claim 7, further comprising the control section performing post-processing, in which the control section converts the computed second feature vector, computes a difference between the converted second feature vector and the first feature vector of the object to be processed, compares the difference with a reference value, and when the difference is greater than the reference value, converts the difference and subtracts the converted difference from the second feature vector.
 10. The method according to claim 7, wherein generating the coupled basis vector comprises multiplying one of the first feature vector and the second feature vector by a scaling factor.
 11. The method according to claim 7, wherein the first index is a standard deviation of a coupling coefficient of the first feature vector and the second index is a standard deviation of a coupling coefficient of the second feature vector.
 12. The method according to claim 7, wherein the first information is a low-resolution image and the second information is a high-resolution image.
 13. A non-transitory computer readable storage medium storing a program for consistently and accurately predicting information executed by an information predicting computer system that includes a learning object storage device that stores objects to be learned, a learning result storage device that stores learning results, and a control section including a processor and connected to an input section, when the program is executed, the control section performs learning processing in which the control section computes a first feature vector and a second feature vector for each object to be learned stored in the learning object storage device, the first feature vector being a feature vector of first information and the second feature vector being a feature vector of second information, the second information being a result of prediction from the first information, generates a coupled basis vector by: generating a coupled vector for each object to be learned by coupling the first feature vector and the second feature vector after equalizing scales of the first feature vector and the second feature vector, and performing machine learning using the coupled vectors of all of the objects to be learned, so as to generate a coupled basis vector, and stores the coupled basis vector in the learning result storage device, and prediction processing, in which the control section computes a first feature vector of an object to be processed that is designated by the input section, and computes a coupling coefficient of the first feature vector of the object to be processed, and using the coupling coefficient of the first feature vector, and the coupled basis vector that is stored in the learning result storage device, computes a second feature vector of the object to be processed.
 14. The non-transitory computer readable storage medium according to claim 13, wherein the control section is configured to generate the first feature vector by dividing a first coefficient, which represents the first information by a first basis vector, by a first index that represents variation of the first information, and to generate the second feature vector by dividing a second coefficient, which represents the second information by a second basis vector, by a second index that represents variation of the second information.
 15. The non-transitory computer readable storage medium according to claim 13, wherein the control section is configured to perform post-processing, in which the control section converts the computed second feature vector, computes a difference between the converted second feature vector and the first feature vector of the object to be processed, compares the difference with a reference value, and when the difference is greater than the reference value, converts the difference and subtracts the converted difference from the second feature vector.
 16. The non-transitory computer readable storage medium according to claim 13, wherein generating the coupled basis vector comprises multiplying one of the first feature vector and the second feature vector by a scaling factor.
 17. The non-transitory computer readable storage medium according to claim 13, wherein the first index is a standard deviation of a coupling coefficient of the first feature vector and the second index is a standard deviation of a coupling coefficient of the second feature vector.
 18. The non-transitory computer readable storage medium according to claim 13, wherein the first information is a low-resolution image and the second information is a high-resolution image. 